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Abstract: An important statistical inference problem in sequential analysis 
is the construction of confidence intervals following sequential tests, to which 
Michael Woodroofe has made fundamental contributions. This paper reviews 
Woodroofe's method and other approaches in the literature. In particular it 
shows how a bias-corrected pivot originally introduced by Woodroofe can be 
used as an improved root for sequential bootstrap confidence intervals. 



1. Introduction and overview 

Estimation following sequential tests is an important but difficult problem in se- 
quential analysis. When the sample size is not fixed in advance but is a random 
variable T that depends on the data collected so far, the sample moments and 
maximum likelihood estimates of population parameters can have substantial bias. 
For example, while the sample mean X n is an unbiased estimate of the population 
mean fi based on a sample of n i.i.d. observations X\, . . . , X n , Xt is clearly biased 
upwards if T is the first time when S n — nX n exceeds some threshold. How can 
one correct for the bias due to data-dependent sample size and how should one con- 
struct a confidence interval for /x? Following Siegmund's seminal paper [lei ] on this 
problem, there have been many important developments in the literature, among 
which are the fundamental contributions of Woodroofe and his collaborators. 

The simplest method to construct a confidence interval for the population mean 
[i is to use the naive normal approximation that treats Vt(Xt — A*) /(Jt as approx- 
imately standard normal, where is a consistent estimate of Var(Ai). Although 
one can justify the normal approximation by appealing to Anscombe's theorem |l[ 
when 

(1.1) T/a^c as a — > oo 

for some nonrandom constant c > and design parameter a > 0, the normal 
approximation essentially treats T as nonrandom and has been found to be unsat- 
isfactory in practice. This led Siegmund to develop exact methods for constructing 
confidence intervals for [i in the normal case by using a certain ordering of the 
sample space of (T, St) when T is the stopping time of a repeated significance test. 
Earlier Armitage used Q numerical methods to evaluate exact confidence inter- 
vals for a Bernoulli parameter following sequential tests. Siegmund's approach was 
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subsequently extended to other stopping rules and to alternative orderings of the 
sample space by Tsiatis, Rosner and Mehta[20|, Chang and O'Brien [5j, Rosncr 
and Tsiatis [17| and Emerson and Fleming [121 ]. 

Woodroofe [22j introduced "very weak" asymptotic expansions to correct for 
Anscombe's normal approximation in a one-parameter exponential family of densi- 
ties fe(x) — e 8x ~ , f'( e ) with natural parameter space 0. Denoting the stopping rule 
T by T a to indicate its dependence on a, he strengthened (1.1) into 

(1.2a) lim E 9 \a/T a - n(9)\ = for a.e. 9 G 6°, 

a—*oo 

lim a / P 9 {T a < ar]c} dO = 

(1.2b) a ^°° Jc 

for some r\c > and every compact C C 0°, 

where 0° denotes the interior of and k : 0° — > R is continuous. Let ^ = 
ip(9) and a 2 = ip(9). Let 0„ be the maximum likelihood estimate of 9 based on 
Xi, . . . ,X n , and fi n — ip(9 n ) — X n , a 2 — ip(9 n ). Consider a Bayesian model in 
which 9 has a twice continuously differentiable prior density function £ with compact 
support [0 O , 6>i] C 0°. With T — T a satisfying (1.2a,b), Woodroofe [H] developed 
an asymptotic expansion for 

(1.3) J P e {Vf(X T - n)/or < c + b(9 T )/VT}£{9) d9, 

where 6 is a piecewise continuous function on 0°, and used it to construct a confi- 
dence interval / for /i whose integrated coverage error 

(1.4) fPe{fi^m(9)de 

differs from the nominal value 2a by o(a _1 ). Subsequently, Woodroofe 23|, 25l 



showed how a version of Stein's identity [19( could be used to derive expressions for 
posterior expectations from which asymptotic expansions for (1.3) could be readily 
guessed. Moreover, for normal Xi with known variance 1, Woodroofe [25| made use 
of these expansions to derive bias-corrected and renormalized pivots of the form 

(1.5) i?(/i) = {VT(X T - M ) - T- 1 / 2 6(A > T )}/{1 + (2T)- 1 b 2 (X T )}, 



where b(fi) — {y/ k(h))' / y/ k{ii) — k' (p)/ {2n(p)}, in which k is given by (1.2a), 
noting that /i = ip(9) = 9 in the normal case. 

Instead of using the Wald-type statistic VT(Xt — At) /err as in (1.3), Coad and 
Woodroofe Q and Weng and Woodroofe [2l| considered confidence intervals based 
on signed-root likelihood ratio statistics, for which they developed very weak ex- 
pansions leading to renormalized and bias-corrected signed-root likelihood ratio 
statistics as pivotal quantities. Woodroofe and his collaborators have also devel- 
oped very weak asymptotic expansions to construct confidence sets in other se- 
quential/adaptive experiments; see d, [3, 23, 24, 27, 28|. 



The preceding methods assume parametric models, and more precisely, exponen- 
tial families. For samples of fixed size, an important methodology for constructing 
confidence intervals without distributional assumptions is Efron's bootstrap method 



The bootstrap methodology can be extended as follows from the fixed sam- 
ple size case to the case where the sample size is determined by a stopping rule 
T. Let X\, Xi, ... be i.i.d. random variables with a common unknown distribution 
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function F. Given a randomly stopped sample (X\, . . . , Xt), let F denote the em- 
pirical distribution that puts probability mass 1/T at each of the sample values 
Xi. Let Xi,X%, ... be i.i.d. random variables with common distribution F and let 
T* denote the corresponding stopping time for the sequence {X*}. The sequential 
bootstrap sample (X£, . . . ,X T ,) can be used to construct confidence intervals as in 
the nonsequential case, and Chuang and Lai [|| have studied the coverage probabil- 
ities of these bootstrap confidence intervals in the setting where T is the stopping 
rule of a group sequential test, for which stopping can only occur at a prespecified 
subset {ni, . . . , tik } of positive integers, and have shown that the bootstrap method 
does not yield reliable confidence intervals because \^T(Xt — h)/&t is no longer 
an approximate pivot. There are "pockets" of the parameter space where T/a has 
a nondegenerate limiting distribution that varies with [i, thus violating (1.1) and 
making the distribution of ^/T(Xt — \i) highly dependent on \i even when the Xi 
are normal with known unit variance. This difficulty can be resolved by replacing F, 
from which the bootstrap method resamples, by a more versatile resampling family 
F^. Specifically, assuming Var(JTj) = 1, the unknown distribution G of Xi — \x can 
be estimated by the empirical distribution G of Xi — Xt- Let = G(- — fi) so 

that F^ has mean fj,, and let u a {n) be the a-quantile of the sampling distribution 
of Vt*(X t , — fi), where the X* are i.i.d. random variables from F^. By analogy 
with the exact confidence set {/i : u a (/i) < \/T{Xt — fj) < iti- a (/x)} proposed by 



Rosner and Tsiatis [171 ] when the Xi are normal, for which the quantiles u a (/i) and 
iti_ a (/i) of the sampling distribution of ^/T(Xt — (j) can be determined by recursive 
numerical integration, Chuang and Lai Q define the "hybrid" confidence set 



as a hybrid of the exact and bootstrap methods. The hybrid confidence set is shown 
to be second-order accurate by making use of an Edgeworth expansion involving a 
fc-variate normal distribution. 

This derivation of second-order accuracy requires k to be fixed and breaks down 
in the case of fully sequential (instead of group sequential) procedures, for which 
Edgeworth-type expansions are considerably more complicated and involve, be- 
sides the usual cumulants of Xi (or X*), certain fluctuation-theoretic quantities 
that are related to the random walk {S n } (or {<S**}), as shown by Woodroofe and 
Keener (2t| and Lai and Wang On the other hand, (1.1) is usually satisfied 
by fully sequential truncated tests such as those studied by Woodroofe and his 
collaborators. Although this implies that VT(Xt — fi) is an asymptotic pivot in 
these fully sequential settings when T becomes infinite, the finite-sample behavior 
of VT(X T - fj) still varies substantially with fi, as will be shown in the simulation 
studies in Section 3. In the context of exponential families, the asymptotic theory 
and the numerical studies of Woodroofe and his collaborators, however, suggest 
that suitable bias correction of \/T(Xt — t 1 ) can improve its pivotal nature sub- 
stantially, making the sampling distribution much less dependent on /i. In Section 
2 we first develop bias-corrected pivots in a nonparametric setting and then make 
use of them to construct nonparametric bootstrap confidence intervals. Numerical 
results are given in Section 3, where we also compare the different approaches to 
constructing confidence intervals following fully sequential tests. Further discussion 
of these results and some concluding remarks are given in Section 4. 




(1.6) 



{fi : u a (jj,) < VT(X T - n) <u 



: i-a(M)} 
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2. Bias correction for a modified pivot and bootstrap confidence 
intervals under optional stopping 

In this section we first review Woodroofe's pivot [2^, [26| that corrects VT(Xt — (J.) 
for optional stopping in the case of a normal population with unknown mean [i and 
known variance 1. We then extend this method to nonparametric problems in which 
the normal mean fj, is replaced by smooth functions of mean vectors of possibly non- 
normal populations. Note that the quantity of interest here is an approximate pivot 
for constructing confidence intervals, rather than point estimates of y, for which 
Siegmund Emerson and Fleming and Liu and Hall have introduced 
bias-corrected or unbiased estimators following sequential tests. 
Suppose the stopping rule is of the form 

(2.1a) T = min{no(a), max(t a , 71.1(a))}, 

where no (a) ~ a/eo and n\(a) ~ o/ei, with < eo < ei, and 

(2.1b) t a = inf{n > 1 : ng{S n /n) > a}, 

in which g is continuously differentiable. A naive pivot is 

(2.2) R (n) = VT(X T - (j,). 

Whereas ^/n(X n — fi) has mean and variance 1 for a fixed sample size n, optional 

stopping affects the first two moments of \/T(Xt — /i). First note that a/t a -Z- g(jj) 

p 1 — ii 

and therefore a/T a — > k(/i) '■— max{eo, min(g(/i), ei)}. As shown by Woodroofe [25j, 

(2.3) ERo(n) = a-^ 2 [(d/d^) K y 2 (fx)] = {n^/a) 1 ^), 

where b{y) = [(d/dfi)K 1 ^ 2 (fj,)]/K 1 ^ 2 (p) = k((j,)/{2k((j,)}. This suggests the bias- 
corrected pivot 

(2.4) R 1 ( f x) =R ^)-T- 1 ' 2 b{X T ) 

as an improvement over (2.2). Moreover, Woodroofe [22| has shown that 

(2.5) P(Ri(v) <x) = $(x) - (2a)- 1 x<j)(x)[(d/dfi)K 1 / 2 (fi)} 2 

in a very weak sense, i.e., the integral of the left-hand side of (2.5) with respect 
to dfj, has an asymptotic expansion given by that of the right hand side (see 
(1.3)), where <j> and $ denote the standard normal density and distribution function, 
respectively. This in turn yields 

(2.6) + {2a)- 1 x 2 {x 2 - l)ct)(x)[(d/d^)K 1/2 (n)} 2 dx 

= l + a- 1 [(d/dfi)K 1 / 2 {fi)} 2 . 

Since {ER 2 ^)} 1 / 2 = 1 + (2a)- 1 n(y)b 2 ! (/i) by (2.6) and since T" 1 = K.{p)/a, these 
calculations led Woodroofe [23] to the approximate pivot (1.5). 

We next remove the assumption of normality on the X; which we also extend to d- 
dimensional vectors. Instead of the mean vector /x, we consider more general smooth 
functions h of fi while the stopping time T is still assumed to be of the form (2.1a,b). 
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Let X, Xi,X2, ... be i.i.d. rfx 1 random vectors with EX = [i, Cov(X) — V and 
E\ \X\ \ r < oo for some r > 3. Let h : R d — > R be twice continuously differentiable in 
some neighborhood of /i. Consider a stopping rule T of the form (2.1a,b), in which 
g : R d — ► R is continuously differentiable in some neighborhood of /i. Suppose 
e o < <7(m) < e i- Then application of the strong law of large numbers in conjunction 
with Taylor's theorem yields 

VT{h(X T ) - h{n)} = VT(S7h{fi))'{X T -n) + VT{X T - //)'V 2 /^)(A T - fi)/2 

(2.7) = -Lg^(S T /T)(S T pT)'Vh{p) 

v a 

+ ~^={T(X T - ri'V 2 h^){X T - M )}, 
2v 1 

in which the last approximate equality follows from Tg(ST/T) = a (ignoring over- 
shoot) so that VT = V^/.9 1/2 (5 T /T) = {a/g(n)} 1/2 . By Wald's lemma, E{g 1 / 2 (/j,)x 
(S T - nT)'Vh(n)} = 0. Moreover, 

(2.8) g 1/2 (S T /T) - g^fjt) = {(V S (/x))'(5 T - T ^)} / {2g^)T} . 

By Anscombe's theorem [l], VT(X T - fi) = (S T - fiT)/VT has a limiting N(0, V) 
distribution. Combining (2.7) with (2.8) and taking expectations, it can be shown 
by uniform integrability arguments that 



(2 9) E W T i h (X T ) - h(fj)}] 

The second term on the right-hand side of (2.9) follows from E(Z'AZ) — tr(AV) 
if A is a nonrandom matrix and Z is a random vector with E(ZZ') = V. 

The difficult part of the proof of (2.9) lies in the technical arguments related to 
uniform integrability. For the case h(x) — x, Aras and Woodroofe Q have provided 
such arguments to develop asymptotic expansions for the first four moments of 
St/T. Particularly relevant to our present problem are their Propositions 1, 2 and 
Section 5, which we can modify and refine to prove (2.9) when eo < g(fJ-) < ei- The 
details are omitted here. For g(p) < eo (or g(p) > e%), stopping occurs at no (a) (or 
ni(a)) with probability approaching 1 and uniform integrability can again be used 
to show that 

(2.10) E[VT{h(X T ) - h(fx)}} = ±{m{a))- l lhT{V 2 h{v)V) + o( fl - 1/2 ), 

with i = or 1 according as g(fJ-) < eo or g(fi) > e\. Since k([i) = max{eo, 
min( ff (^), £l )}, Vk 1 / 2 ^) = ±VgQi)/(gQjt))V 3 if e < g(ji) < d, and Vk 1 /^) = 
if g(fi) < eo or > t\. Recalling that l/no(/i) ~ e o/ a and l/ni(/x) ~ ei/a, we 
can combine (2.9) and (2.10) into 



(2.11) E[VT{h(X T ) - h(M)}] = 6( M , V){K^)/afl 2 + o^ 1 / 2 ), 
where 

(2.12) b(jt, V) = (VK 1/2 (/i))'yV/i(^)/K 1/2 (^) + tr(V 2 h(n)V)/2. 

For the special case d = 1, h(fi) = a and V = 1, b(fj,, V) — [(d/dfj,)K 1 ^ 2 (fi)]/K 1 ^ 2 (^i), 
which agrees with Woodroofe's [25| approximation for E{s/T(Xt — (J-)} derived 
from very weak asymptotic expansions for normal X. 
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Since fj, is unknown, replacing fi by Xt and n(fi)/a by 1/T in the last term of 
(2.3) leads to Woodroofe's [25| bias-corrected pivot in (2.4). In the present 

nonparametric setting, V is typically also unknown and has to be estimated to 
define both the naive pivot Rq{h) and its bias-corrected version i?i(/i). Using the 
consistent estimates 

T 

(2.13) V T = Y,( x * - X T )(Xi - X T )'/(T - 1 ), g% = (Vh(X T ))'V T Vh(X T ) 

z=l 

of V and the asymptotic variance a 2 := (Vh(/i)yWh(fi) of ^/T{h(XT) — h(/i)}, 
define 

(2.14) Ro(fi) = VT{h(X T ) - h(ji)}/ar, 



(2.15) Rxifj,) = WT{h{X T )~h{^)}~T- l / 2 b{X T ,VT)yd Tl 

where b(p,, V) is defined in (2.12). 

For the case of normal mean with known variance 1, Woodroofe 25| further 
refined Ri (/i) by scaling it with an asymptotic approximation to the standard devi- 
ation that he derived by very weak expansions; see (2.6) and (1-5). In the nonpara- 
metric setting with unknown covariance matrix V considered here, such refinements 
are considerably much more complicated. In particular, better approximations to 
the asymptotic standard error than ctt (which is derived by linearizing h around /i) 
are needed. We therefore forgo such refinements and simply use the bias-corrected 
pivot Ri(fi) instead. The following example, which deals with the same testing 
problem as that considered in Woodroofe's [25| simulation study, shows that there 
is not much loss in the quality of the normal approximation to the distribution of 
i?i(/i) in comparison with that of R{fJ-). 

Example 1. Suppose X is normal with mean \x and known variance 1. Let 

g(x) = (2S)- 1 {S 2 + x 2 )l {lxl < s} + \x\l { \ x \ >s} , 

which is symmetric and continuously differentiable, with g — x/5 for < x < S 
and g = 1 for x > 6. Take a = 9, no(a) = 72 and ni(a) = 1 in (2.1a), as in Section 



4 of [25(. Table 1 gives the a-quantiles of i?(/Lt), -Ro(a*) and Ri(p-), respectively, 
computed from 10000 simulations, over different values of /i ranging from to 1 as 
in Woodroofe's study. In this known variance setting, i?(/i), -Ro(m) an d Ri(l^) are 
defined by (1.5), (2.2) and (2.4). Without assuming the variance to be known, we 
can use the version (2.15) for i?i(/i), which we denote by i? 1 ^(/x) to indicate that 

the variance V(— a 2 ) is replaced by the sample variance u\. Table 1 also gives 
the a-quantile of i? 1 ^(/i) and the standard normal quantiles z a for comparison. It 
shows that when a < 10% or a > 90%, q a (R) does not differ much from z a and 
does not change much as /i varies between and 1. The normal approximation is 
somewhat worse for q a (Ri) or q a (R 1 ^) which, however, still does not change much 
as fi varies between and 1. The normal approximation deteriorates substantially 
for q a (Ro); moreover, q a (Ro) is also markedly more variable with /i, making it less 
"pivotal" . 

Since the stopping rule T is assumed to be of the form (2.1a,b), T/a 1/k(/i) 
and (1.1) is clearly satisfied. Therefore, by Anscombe's theorem, Ro(fi) is an asymp- 
totic pivot and so is i?i(/z) that introduces a correction of the order O p (a~ 1//2 ) for 



o 



Table 1. 





Quantiles z a ,q a (R), 


9a(Rl),9a(R 1 ~) 


and q a (Ro) 


of the standard normal distribution, R(/i), Ri(^t), R ~(/i) and Rq(/j.), respectively 

1,(7 






a (in %) 


2.5 


5 


10 


20 


50 


80 


90 


95 


97.5 




-1.96 


-1.645 


-1.28 




n 
U 




1 98 
l.Zo 




1.96 














(a) fj, = 












9a (R) 


-1.9327 


-1.6280 


-1.2902 


-0.9053 


0.0035 


0.8878 


1.2832 


1.6265 


1.9880 




g a {Ri) 


-1.9904 


-1.6581 


-1.3344 


-0.9372 


-0.0223 


0.9491 


1.3206 


1.6498 


1.9721 




9a (R -) 


-2.0007 


-1.6929 


-1.3351 


-0.9163 


0.0045 


0.9205 


1.3012 


1.7062 


2.0083 




9a (Ro) 


-2.2449 


-1.9255 


-1.6466 


1 001 Q 
— 1 . ZZLo 


u.uu / o 


1.10 l / 


1 £.A 1 O 




2.2369 










(b) = 0.25 












9a (R) 


-1.9913 


-1.6558 


-1.2373 


-0.8129 


0.0029 


0.8257 


1.2795 


1.6803 


1.9698 




9a(fll) 


-2.0807 


-1.5996 


-1.2508 


-0.8376 


-0.0322 


0.8413 


1.3063 


1.6397 


1.9515 


g 




— 1.9457 


— 1.6106 


— 1.2441 


-0.8274 


-0.0263 


0.8462 


1.3106 


1.6641 


2.0U33 




9a (Ro) 


-1.9320 


-1.4949 


-1.0682 


-0.5817 


0.2862 


1.0911 


1.5139 


1.8418 


2.1377 














(c) fi = 0.5 










o 

3 


9a (R) 


-1.8817 


-1.6496 


-1.2685 


-0.8450 


-0.0091 


0.8208 


1.2448 


1.6156 


1.9321 


0. 


9a (Ri) 


-2.0758 


-1.7250 


-1.3613 


-0.8840 


-0.0235 


0.8333 


1.3260 


1.6396 


1.9818 


p 


9a (R, -) 


-2.0272 


-1.7000 


-1.3314 


-0.8745 


-0.0209 


0.8374 


1.2769 


1.6494 


1.9985 




1 ,(7 

9a (Ro) 


-1.7005 


-1.4107 


-1.0305 


-0.6237 


0.1894 


1.0435 


1.4668 


1.8104 


2.1436 












(d) fj, = 0.75 










e 
a 


9a (R) 


-1.9435 


-1.6190 


-1.3004 


-0.8758 


-0.0188 


0.8284 


1.3224 


1.6094 


1.9519 


3 


9a (Rl) 


-2.0479 


-1.6768 


-1.3027 


-0.8684 


-0.0089 


0.8555 


1.3153 


1.6574 


1.9600 




9a (R ~) 


-2.0041 


-1.7406 


-1.3314 


-0.8787 


-0.0035 


0.8228 


1.2873 


1.6292 


1.9441 




9a (Ro) 


-1.7241 


-1.4270 


-1.1190 


-0.6657 


0.1415 


1.0247 


1.4202 


1.8004 


2.1070 




9a (R) 


-1.9281 


-1.6617 


-1.2650 


-0.8719 


(e) ix = 1 
-0.0257 


0.7956 


1.2505 


1.6780 


1.9335 




9a (Rl) 


-2.0028 


-1.6963 


-1.3056 


-0.8719 


-0.0412 


0.8106 


1.2514 


1.6840 


1.9542 




9a (R, ~) 


-2.0062 


-1.6821 


-1.3314 


-0.8344 


-0.0190 


0.8424 


1.2520 


1.6669 


1.9506 




1,<7 

9a (Ro) 


-1.8202 


-1.5173 


-1.1161 


-0.6909 


0.1150 


0.9394 


1.4222 


1.8004 


2.0165 
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i?o (/i) ■ Although bootstrap confidence intervals based on Rq (fi) or Ri (/x) are there- 
fore asymptotically valid, Example 1 suggests that Ri(n) is more "pivotal" and 
may therefore provide substantial improvements over Ro(p) for the finite-sample 
coverage errors. Simulation studies comparing both types of bootstrap confidence 
intervals are given in Section 3 to confirm this. A theoretical comparison would in- 
volve higher-order asymptotic expansions. While Woodroofe's very weak expansions 
are not applicable to the present nonparametric setting, the much more complicated 
Edgeworth-type expansions of Woodroofe and Keener and Lai and Wang [3] 
can still be applied and will be presented elsewhere. 



3. Numerical comparisons of various confidence intervals following 
sequential tests 

Let T be a stopping rule of the form (2.1a,b). Based on the sample X±,..., Xt of 
random size T, the normal confidence interval 



(3.1) (h(X T ) - z^ a a T /VT, h(X T ) - z a a T /VT) 

simply uses the normal quantiles z a and z\^ a to approximate the corresponding 
quantiles of Rq(/j,), invoking Anscombe's theorem for its asymptotic justification. 
Similarly we can apply the normal approximation to the a- and (1 — a)-quantiles 
of Ri(fJ,), leading to the interval 



(3.2) 



h (X T ) - fo-»5r + T- l/2 b{X T , V T )} 



h(X T ) 



T 

{ZaOT + T-Wb&TtVr)} 



T 



Instead of approximating the quantiles of Rq((j>) or Ri(fJ-) by normal quantiles, 
we can approximate them by the quantiles of the sequential bootstrap sample 
(X{, . . . , XrL,) described in the penultimate paragraph of Section 1, leading to the 
bootstrap confidence intervals based on Rq(i-i) or i?i(/i). 

The second paragraph of Section 1 has reviewed previous works on the exact 
method. As described more generally by Chuang and Lai [7J, p. 2], the exact method 
involves (i) a family of distributions Fg indexed by a real-valued parameter 9 and 
(ii) a statistic r(9; T, X\, . . . , Xt) for every given value of 9, called a root. Let u a (9) 
be the a-quantile of r(9; T, X\, . . . , Xt) under Fg. An exact equal-tailed confidence 
set for 9 with coverage probability 1 — 2a is 

(3.3) {9 : u a {9) < r(9; T, X u . . . , X T ) < u X - a (0)}, 

For the normal mean example (with 9 = fj,) considered by Siegmund (l8j . r(/i; T, 
X\, . . . , Xt) = (T, St), for which he introduced a total ordering to define the p- 
quantile u p (9). An obvious alternative choice is rUt\T, X\, . . . , Xt) — VT(Xt — (J>) 
that has been considered by Rosner and Tsiatis [171 ]. 

The exact method applies only when there are no nuisance parameters. In prac- 
tice, however, not only do parametric models usually involve nuisance parame- 
ters, but one may also have difficulties in coming up with realistic parametric 
models. Without distributional assumptions on X, a 1 — 2a level bootstrap confi- 
dence interval for a functional 9(F) of the distribution F of X, based on the root 
r(9; T,X U ..., X T ), is of the form 

(3.4) {9: u* a < r(9; T, X 1 , . . . , X T ) < uJ.J, 
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where it* is the a-quantile of the distribution of r(9;T*,X*, . . . ,X£„) in which 
9 = 9(F) and {X*, . . . , X T ,) is a bootstrap sample with random size T* drawn 
from the empirical distribution F of (Xi, . . . , Xt)- The bootstrap confidence inter- 
val (3.4) is tantamount to replacing u a and Ui_ a in (3.3) by u* a and u\_ a when 
r(9; T, X\ , . . . , Xt) is an approximate pivot, so that the quantile u* evaluated under 

F can approximate the quantile under the true distribution F. 

The hybrid method mentioned in Section 1 is based on reducing the nonpara- 
metric family T containing F to another family Fg, where 9 = 9(F) is the unknown 
parameter of interest. It is particularly useful in situations where the sampling dis- 
tribution of the root r(0\T,X\, . . . ,Xt) may depend on 9 but is approximately 
constant over {F £ T : 9(F) — 9}, as in group sequential clinical trials studied by 
Chuang and Lai Q and in possibly nonstationary first-order autoregressive models 
considered in Section 5 of Chuang and Lai [2]. Applying the exact method to the 
family {Fg} yields the hybrid confidence set 

(3.5) {9 : u a (9) < r(9; T, X lt . . . , X T ) < ui_a(0)}. 

Note that hybrid resampling is a generalization of bootstrap resampling that uses 
the singleton {F} as the resampling family. 

In the following simulation studies we compare the hybrid confidence interval 
(3.5); the bootstrap confidence intervals Boot(i?o) and Boot(i?i) that use i?o and 
R\ , respectively, as the root in (3.4); their direct normal approximation counterparts 
(3.1) and (3.2), denoted by Normal(i?o) and Normal(i?i), respectively; the exact 
confidence interval (3.3); and Woodroofe's [25| interval, denoted by Normal(-R), 
that uses the normal approximation to the renormalized pivot R(fi) in (1.5) derived 
under the parametric model. The quantiles in the bootstrap (or hybrid) confidence 
intervals are computed from 1000 samples drawn from F (or Fg). 

Example 2. Let Xi,X2, ... be i.i.d. N(/x, 1) random variables and let T be the 
stopping rule of the form (2.1a,b) with g(x) — x 2 /2,a — 4.5, m(a) = 15 and 
n (a) = 75. This corresponds to [4[ repeated significance test (RST) of H : ji = 
that stops sampling at T = inf{n > 15 : \S n \ > 3y/n} A 75. Table 2 gives the 
coverage errors of the upper (U) and the lower (L) confidence bounds for fj, following 
the RST, constructed by the various methods reviewed above with the nominal 
coverage error a — 5%. Each result is based on 10000 simulations. It shows that the 
hybrid confidence limits have coverage errors similar to those of the exact method 
(which should be 5%, with departures from 5% due to the Monte Carlo sampling 
variability). Woodroofe's method using normal approximation for the pivot R also 
works well, except for a case (fj, = 1.2). While the normal confidence interval (3.1) 
using Rq as the pivot has inaccurate coverage for certain values of //, the confidence 
interval (3.2), which uses i?i as the pivot, and the bootstrap confidence intervals 
Boot(i?o) and Boot(i?i) show substantial improvement. 

Example 3. The "exact" method for constructing the confidence interval (3.3) 
requires precise specification of a one-parameter family Fg , which we have assumed 
to be N(0, 1) in Example 2. Woodroofe's pivot R(fi) is also derived under such para- 
metric assumption. On the other hand, the pivots Ro(fi) and R\(p) can be derived 
nonparametrically. Suppose the underlying distribution F is actually a mixture of 
N(//, 1) and fi + (Exp(l) — 1), putting mixing probability 0.2 on N(/i, 1) and 0.8 on 
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the other component of the mixture distribution, where Exp(l) denotes the expo- 
nential distribution with mean 1. Although the mixture distribution still has mean 
fi and variance 1, the skewness of the exponential component adversely affects the 
coverage errors of the "exact" confidence interval that assumes normality and those 
of the confidence intervals based on the normal approximation to R(fJ-), Rq{^) and 
respectively, as shown in Table 3 that uses the same stopping rule as that 
of Example 2, a — 5% and 10000 simulations to compute each coverage error. 
The bootstrap confidence interval Boot(i?i) shows substantial improvement and 
the hybrid confidence interval performs even better. 

Example 4. The stopping rule T in Examples 2 and 3 is associated with the 
RST when the variance is known. In the case of unknown variance, an obvious 
modification is 



where a 2 = n 1 ^™ =1 {X i — X n ) 2 . Note that T is still of the form (2.1a,b), with 
a = 4.5 and g : R x (0, oo) — > [0, oo) defined by 



since ng(£i Xi/n, T,i x f/n) = S 2 J{2na 2 n ). Let /x = EX and ^ = EXi for 



j > 1. Then the covariance matrix of (X, X 2 )' has n% — /i 2 and //4 — fi 2 as its 
diagonal elements, and its off-diagonal elements are both equal to ^ — More- 
over, (Vp)(jU, /i.2) = (mM2i — M 2 /2)'/(a*2 — M 2 ) 2 - To construct confidence intervals for 
/i in the case of unknown variance, the bias-corrected pivot in (2.15) can 

be computed easily by setting V/i(/x, ^2) = (1, 0)' in (2.12), which corresponds to 
MMj M2) — M- Table 4 gives the coverage errors of the confidence limits for fj,, using 
i?o and R%, respectively, and also by bootstrapping R\. The quantiles i(i?o) and 
t(R\) of the ^-distribution with T degrees of freedom for the pivots Rn and i?i are 
used in lieu of normal quantiles, following Woodroofe and Coad [28l . Section 4]. 
Again, a — 5% and each result is based on 10000 simulations. It shows consider- 
able improvement of Boot(i?i) over the confidence limits based on ^-distribution 
approximations for the approximate pivots Rq and R\. 

4. Conclusion 

In their discussion on pp. 33-36 of 0], Woodroofe and Weng have given some 
comparative studies of the hybrid method with the approach based on very weak 
expansions for constructing confidence intervals following group sequential tests. 
Their simulation studies have shown that the expansions "do not work very well 
for the repeated significance tests in Example 1 of the paper" and that "with 
large horizons, however, expansions work very well for triangular tests in which 
g(x) = 5 + \x\." They also raised the issue concerning "robustness with respect 
to the normality assumption." Their approach based on very weak expansions was 
subsequently extended by Morgan [l6| to more general group sequential tests. This 
paper continues their investigation in other directions. First we consider fully se- 
quential instead of group sequential tests. Secondly, we show the robustness of the 
bias correction, which Woodroofe derived by very weak expansions, by rederiving 
it without the parametric assumption. Thirdly, instead of applying directly normal 
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approximation to the bias-corrected pivot, we use it as the root for constructing 
bootstrap confidence intervals. Simple bootstrapping indeed substantially reduces 
the computational cost of the hybrid method. Our conclusion from the comparative 
studies in Section 3 is that at the expense of greater computational cost, the hybrid 
method still provides the most reliable confidence intervals. Moreover, it is also 
much more versatile and can handle complex statistical models, as recently shown 
by Lai and Li |13j for the problem of valid confidence intervals for the regression 
parameter of a proportional hazards model following time-sequential clinical trials 
with censored survival data. 
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